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ABSTRACT 

This study examined the relative efficacy of the Anchorage 
(Alaska) Pre-Algebra Test and the State of Alaska Benchmark in 2 Math 
examination as tools used in the process of recommending grade 6 students for 
grade 7 Pre-Algebra placement. The consequential validity of the tests is 
explored in the context of class placements and grades earned. The Pre-Algebra 
examination was developed as an element of the selection process, since about 
one third of the district's students enroll in Pre-Algebra in grade 7. 

Original cut scores were informally validated through the experience of middle 
school teachers and no need was found to modify the examination or change the 
cut scores. To compare the Anchorage test and the state's test, data were used 
for 3,352 Anchorage students in grade 7 and correlations with the tests' 
results and grade 8 grades overall were also determined. Scores on the 
examinations are highly related with a correlation of greater than 0.8 even 
though the Anchorage examination consists entirely of multiple choice 
questions and the Alaska Benchmark examination includes short-answer and 
extended response questions. Recommendations based on the two examinations 
would be highly similar. The Anchorage examination that was actually used to 
place students was only slightly better in predicting actual placement than 
the Alaska Benchmark examination. Results support the consequential validity 
of either examination for selection of students for Pre-Algebra. (Contains 12 
tables and 3 references.) (SLD) 
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Introduction 



This study examines the relative efficacy of the Anchorage Pre-Algebra Test and the 
State of Alaska Benchmark 2 Math exam as tools used in the process of recommending 
grade 6 students for grade 7 Pre-Algebra placement. The consequential validity of the 
tests is explored in the context of class placements and grades earned. 

Moving students forward to success in advanced mathematics is a goal of the Anchorage 
School District. The Anchorage School Board has called for an increase in the 
percentage of students who complete Algebra by the time they finish grade' 8. and the 
development of Algebra skills in all students (Anchorage Board Goals, 2001-2002). This 
goal is consistent with Alaska State Standards in Mathematics and the inclusion of 
Algebra problems on the Alaska High School Graduation Examination (Alaska 
Department of Education. 2000). 

The high standards set by the State of Alaska and the ambitious goals of the Anchorage 
School Board have encouraged the development of a mathematics sequence that allows 
and encourages students to enter Pre-Algebra in grade 7 and to complete Algebra I in 
grade 8. Not all students, however, are ready for the cognitive demands of Pre-Algebra 
in grade 7 and the Anchorage School District is committed to the ongoing success of 
students as they move through the mathematics sequence to reach the level of proficiency 
needed for high school graduation. 

The Anchorage School District Pre-Algebra Exam was developed by a group of 
Anchorage Mathematics teachers with the support of the Anchorage Assessment and 
evaluation department to serve as one element in the selection process of students for Pre- 
Algebra in grade 7. The 40-item examination includes questions that were selected both 
to reflect the Anchorage 6 lh grade curriculum and to assess math reasoning and problem 
solving. The multiple choice test was administered to a group of students to establish a 
performance baseline in 1998 and a committee of teachers used a modified Angoff 
process to establish cuts scores. The exam is administered in late April and early May. 

The original cut scores were informally validated through the experience of middle 
school teachers and no need was found to either modify the exam or change the cut 
scores. There is some anecdotal evidence that the exam became a motivation for 6 lh 
grade teachers to work through the entire 6 th grade curriculum to assure that students 
would be prepared for the end-of-vear examination. 
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The State of Alaska introduced a new Comprehensive Student Assessment System in 
2000. The system is consists of standards-based exams for grades 3. 6. and 8 followed by 
the High School Graduation Qualification exam at the end of grade 10. The Mathematics 
Examinations are keyed to the Alaska State Standards and include multiple choice, short 
answer, and one extended response items. The Alaska Benchmark exam is administered 
in early March. 

The examinations differ in three salient features. 

• The Anchorage examination was developed to reflect the Anchorage 6 th grade 
curriculum and includes some questions selected to show underlying 
mathematical ability 7 . 

• The Alaska examination was developed to reflect the Alaska State Standards and 
includes a variety of response formats. 

• The Alaska examination is given approximately six weeks earlier than the 
Anchorage examination. 

This study examines the relationship of the two examinations given to grade 6 students in 
the spring of 2001 and the math grades earned by the same students in grade 7 during the 
2001-2002 school year. All students who had examination scores and grades reported at 
the end of the first semester on Anchorage Report Cards are included. 

Equipercentile equating was done to match the two tests. The cut scores established for 
the Anchorage Pre-Algebra test were applied based on the percentiles established for all 
students taking both tests. All analysis was completed using the Statistical Package for 
the Social Sciences for Windows version 1 1.0 (SPSS, 2001). 

Primary Research Question 

The primary research question is: To what extent do the Anchorage Pre-Algebra and 
Alaska State Benchmark Exam result in the same recommendations as to the placement 
of students. 

The secondary 7 research questions is: To what extent do the Anchorage Pre-Algebra and 
Alaska State Benchmark Exam correctly predict the actual placement and performance of 
students in their 7 th grade mathematics courses. 
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Findings 



Who were the students and how were they doing in mathematics at the end of the 
first semester of grade 7 mathematics? 

All grade 7 students are assigned to some mathematics course. For the purposes of this 
study students were organized into three groups based on course numbers. Group 1 is 
mostly made up of students enrolled in Math 7 though some in pre-Math 7 level courses 
are included. Group 2 includes only students enrolled in Pre-Algebra. Group 3 is small 
and includes students enrolled in Algebra 1 and Geometry. About 1/3 of Anchorage 
students enroll in Pre-Algebra and are on track for Algebra 1 in grade 8. 

Table 1 

Grade 7 Math Enrollments 
First Semester 2001-2002 



Course Type 




Students 

Enrolled 


Percent 

Enrolled 










Math 7 & Below 




2,221 


67% 


Pre-Algebra 




1,042 


31% 


Algebra 1 & Above 




76 


02% 










Total 




3,352 


100% 



Grades are complex indicators of performance that are used by teachers to both reflect 
some objective level of performance and to reward students for their effort. The selection 
of students for the various levels of courses and the differences in course content mean, 
of course, that students in the three groups are doing different levels of work to attain 
grades. While the content may be easier to master in Math 7. the Math 7 student may- 
have to put in as much hard work to get an "A’ ? grade as the gifted student in Algebra 1 . 
In some schools, counselors report that they feel that students with grades of “D” or i£ F’' 
should not be advanced to Algebra from Pre-Algebra as the low letter grades demonstrate 
that the student lacks sufficient mastery of the content to be successful in an even more 
difficult course. 

Table 2 

Grades Earned All Grade 7 Math Courses 
First Semester 2001-2002 



Grade 


Number 

Combined 


Percent 

Combined 


A 


772 


23% 


B 


987 


29% 


C 


706 


21% 


D 


368 


11% 


F 


281 


08% 
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How do the Anchorage Pre-Algebra and the State Benchmark 2 tests relate to each 
other? 

A commonly used statistical indicator of association is the correlation coefficient. This 
indicator relates the relative position of scores when ordered from highest to lowest. A 
strong correlation of .7 or better indicates a strong relationship. In the social sciences, a 
correlation of .3 or better is generally significant and considered to show a real 
relationship. In measurement, a correlation of .9 or better generally means that two tests 
are very strongly related. 

The overall correlation of .81 found between the Anchorage Pre-Algebra test and State 
Benchmark 2 indicates that the tests produce highly similar scores but are not a perfect 
match. A .81 correlation between the two tests is made more substantial when you 
consider the differences in test format and the time of test administration. 

The Anchorage Pre-Algebra and State Benchmark 2 Math tests organize student scores in 
almost the same order from high to low. This makes the two tests almost equivalent as 
instruments for making a recommendation about the placement of students based on their 
mathematics performance. 

How do the Anchorage Pre-Algebra and State Benchmark 2 tests relate to the 
grades earned by students in grade 8? 

Grades and scores on tests based on academic knowledge are generally not as strongly 
associated as scores on two similar tests. While tests are good measures of knowledge 
and the ability to perform tasks such as solving a specific problem, thev are very' different 
from grades. Grades are generally issued to students based on performance over time and 
include many elements beyond the knowledge and skill measured on paper and pencil 
tests. For example, teachers often take into account effort, homework completed, and 
class participation. A student who makes a substantial effort in class will often be 
rewarded with a grade that might not be expected from that student's test score. 
Correlations between grades and test scores are often in the .25 to .40 range. The more of 
a grade that is based on test-like performances, the higher the association. 

For the purposes of this analysis, letter grades on Anchorage report cards were assigned 
points. A —5. B —4. C = "D' = 2. and "F ’ = 1. No additional points were given 

to differentiate grades earned in honors or advanced placement classes. The average 
Grade Point across all 7 lh grade math courses was 3.26 or just a little above a “C.” 

The overall correlation between test scores on both the Anchorage Pre-Algebra test and 
the State of Alaska Benchmark 2 exam and GPA is .32. This reflects a moderate but 
statistically significant relationship. 
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Table 3 

Correlation of Grade 6 Test Scores and Grade 7 Math Grades 



Test 


Correlation 


Significance 
Greater Than 








State Benchmark & ASD 
Pre-Algebra Test Correlation 


.81 


.01 








Test and GPA Correlations 












ASD Pre-Algebra & GPA 


.32 


.01 


State Benchmark Total & GPA 


.32 


.01 


State Standard 1 


.28 


.01 


State Standard 2 


.28 


.01 


State Standard 3 


.31 


.01 


State Standard 4 


.31 


.01 


State Standard 5 


.31 


.01 


State Standard 6 


.32 


.01 


State Standard 7 


.32 


.01 



Those Alaska State Standards that deal with more advanced skills (Standards 3 through 
7) are slightly more closely related to grade 7 performance than those standards that deal 
with number sense and arithmetic. 

What is the pattern of student recommendations for placement that were derived 
from the Anchorage Pre-Algebra tests and would be derived from the Alaska 
Benchmark 2 exam? 

The comparison of the classifications of the two exams is based on equating the Alaska 
Benchmark 2 exam with the Anchorage Pre-Algebra exam. After teachers set the cut 
score on the Anchorage exam, it was decided to report results in three rather than two 
categories to reflect the error found in any test. This assures that the students just above 
and just below the cut score were given careful consideration for inclusion in Pre-Algebra 
classes based on the recommendations of their teachers. Students are classified as Pre- 
Algebra, review, or Math 7. 

The standard error of the State Benchmark exam score is smaller than that found for the 
Anchorage Pre-Algebra exam. If the cut score setting process were done with a group of 
Anchorage teachers setting a cut score on the Benchmark 2 exam, there would be fewer 
students in the review category. By projecting the existing Anchorage Pre-Algebra test 
cut scores onto the State Benchmark Exam, more students are included in the review 
category than would be if the State Benchmark Exam standard error were used. 
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There might also be some difference in student effort expected on the two tests. While 
the State of Alaska Benchmark 2 has no direct consequences for the 6 th grade student, the 
Pre-Algebra test is known by the student to affect grade 7 math placements. The 
common experience with student tests is that there is a motivational effect when the 
student feels that the test scores will affect him. If the Benchmark 2 math test was sued 
for Pre-Algebra placement, one could safely assume a positive motivational effect that 
could raise overall Benchmark 2 math test performance for Anchorage. 



Table 4 

Grade 6 Pre-Algebra Recommendations 
Anchorage Pre-Algebra Placement Test 



Group 


Number 


Percent 


Math 7 


1,848 


57% 


Review 


523 


16% 


Pre-Algebra 


848 


26% 



Table 5 

Projected Grade 6 Pre-Algebra Recommendations 
State of Alaska Benchmark 2 Math Exam 



Group 


Number 


Percent 


Math 7 


1.774 


55% 


Review 


482 


15% 


Pre- Algebra 


962 


30% 



Without a change in the cut scores, it appears that there would be a slight increase in the 
proportion of students recommended for Pre-Algebra if the State of Alaska Benchmark 2 
math test were used rather than the Anchorage Pre-Algebra exam. 

How do actual placements relate to the recommended placements? 

When Pre-Algebra and Algebra+ placements are combined, accuracy on the State 
Benchmark 2 math test is 77%. Accuracy on the ASD Anchorage Pre-Algebra test is 
69%. The similar figures for Math 7 placement are 82% for the State of Alaska test and 
77% for the Anchorage Pre-Algebra test. 

This high level of agreement between scores and actual placement is a strong 
recommendation for both of the tests and confirms of the utility of the Anchorage test 
even if the State Benchmark 2 exam has slightly better agreement between the test 
performance and the actual classification of students. 
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Table 6 

Anchorage Pre- Algebra Placement Test 
Actual vs. Test Recommended Placement 



Recommended/Actual 


Math 7 


Pre-Algebra 


Algebra + 










Math 7 


1,625 


140 


0 




77% 


14% 


00% 


Review 


297 


184 


1 




14% 


18% 


' 02% 


Pre-Algebra 


186 


671 


69 




09% 


67% 


98% 


Total (100%) 


2,108 


995 


70 



Table 7 

Alaska Benchmark 2 Math Test 
Actual vs. Test Recommended Placement 



Recommended/Actual 


Math 7 


Pre-Algebra 


Algebra + 










Math 7 


1,723 


113 


0 




82% 


11% 


00% 


Review 


269 


254 


0 




13% 


25% 


00% 


Pre-Algebra 


117 


654 


76 




05% 


64% 


100% 


Total (100%) 


2,109 


1.021 


76 



How do predicted placements relate to actual placements? 

A statistical procedure, multiple discriminate analysis, allows the grouping of individuals 
based on a variable such as a test score to see how well the actual placement matched the 
placement that would be done if only the score had been used for placement. This 
analysis is slightly complicated by the fact that the Anchorage Pre-Algebra test was only 
used to consider placement in Math 7 or Pre-Algebra when some students went into 
Algebra I or Geometry. In most cases, students who were assigned to the more advanced 
classes had previously taken some advanced mathematics course work and possessed 
substantial math ability. These students are included as a separate group in the tables 
below though their test scores would have qualified them for Pre-Algebra. 
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Table 8 

Accuracy of Predicted Placements 

ASD Pre-Algebra Test Performance used to Predict Math Placements 



Actual/Predicted 


Math 7 


Pre-Algebra 


Algebra + 










Math 7 


1704. 


301 


4 




85% 


1 5% 


.02% ■ 










Pre-Algebra 


143 


662 


171 




15% 


68% 


18% 










Algebra + 


0 


20 


50 




00% 


29% 


71% 



Table 9 

Overall Accuracy of Predicted Placement 
ASD Pre-Algebra Test and State Benchmark 2 Path Test 



Test/Course Placement 


Math 7 


Pre-Algebra 








Anchorage Pre-Algebra 


72% 


78% 


State Benchmark 2 Math Test 


71% 


69% 



The Anchorage Pre-Algebra test that was actually used as part of the assignment process 
only has a slight advantage over the State Benchmark 2 Math Test when used to predict 
the actual math class placement of students. This is a demonstration of the substantial 
equivalence of the tw'o tests in their relationship to the math placement of students in 
2001 - 2002 . 
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How strong is the relationship between the State and Anchorage test scores and the 
grades that the students earn in the three levels of courses? 

The general expectation here, based on the overall test to grade point average, is that the 
overall relationship would be about .28. A stronger relationship would suggest that 
grades have more to do with math performance and a weaker relationship would suggest 
that grades may have more to do with effort and class participation than math 
performance. 

The differences displayed in Table 8 are notable and significant. The low weak 
relationship (.16 and .17) found between grades and test scores for Math 7 may be due in 
part to the inclusion of a few pre-Math 7 special education classes in the Math 7 group 
while almost no Special Education students are found in either the Pre-Algebra or 
Algebra 1+ groups. 



Table 10 

Correlation of Grade 6 Test Scores to Grade 7 First Semester Grades 
Anchorage Grade 7 Math Courses 



Test/Course 


Math 7 


Pre-Algebra 


Algebra + 










Anchorage Pre-Algebra 


.16 


.27 


.34 


State Benchmark 2 Math 


.17 


.24 


.36 



The correlations mask notable differences in average scores among of students who earn 
the various letter grades in the three levels of math classes. Table 9 shows the average 
scores on the Anchorage Pre-Algebra Test and on the State Benchmark 2 Math tests. The 
consistent pattern of scores supports the validity of the two tests as indicators of math 
outcomes for students. The differences in the average scores related to the three levels of 
classes are statistically significant in all cases for both tests. 

Table 1 1 

Average Scores by Course Type 



Test/Course 


Math 7 


Pre-Algebra 


Algebra+ 


ASD Pre-Algebra 


17.1 


28.0 


34.3 


N = 3,337 


N = 2.109 


N = 1.052 


N = 76 










State Benchmark 


326 


434 


503 


N = 3,273 


N = 2.108 


z 

II 

SO 

SO 

fyi 


o 

r- 

II 

Z 



Table 12 further demonstrates the differences by looking at the actual letter grades earned 
by students and the associated average test scores on the ASD Pre-Algebra Test and the 
State of Alaska Benchmark 2 Math Test. The pattern of average scores related to letter 
grades for each type of course shows both the differences in performance levels of 




12 



ii 



students assigned to the classes and the differences between letter grades earned. While 
the letter grades are never the consequence of the test scores, it is clear that the 
association of grades and performance is strong and consistent. 



Table 12 

Average Performance 

Anchorage Pre-Algebra Test and State Benchmark 2 Math Tests 
Broken Down by Letter Grade in Three Levels of Math Courses 



Grade 


Course 


Average 
Anchorage 
Test Score 


Number 

Of 

Students 


Average 
State Math 
Test Score 


Number 

Of 

Students 


A 




25.3 


751 


409.1 


727 




Math 7 


19.7 




354.6 






Pre-Algebra 


29.8 




452.6 






Algebra + 


35.1 




523.1 
















B 




21.7 


959 


373 


941 




Math 7 


17.6 




333.4 






Pre-Algebra 


28.0 




435.0 






Algebra + 


33.7 




523.1 
















C 




19.4 


680 


343.0 


677 




Math 7 


16.1 




315.6 






Pre- Algebra 

iK, 


26.2 




414.8 






Algebra + 


32.4 




477.0 
















D 




17.8 


349 


334.0 


357 




Math 7 


15.3 




311.8 






Pre-Algebra 


26.3 




410.2 






Algebra '+ 


37.0 




418.0 
















F 




16.2 


263 


318.0 


264 




Math 7 


15.1 




305.6 






Pre- 

Algebra 


24.1 




402.2 






Algebra + 


33.0 




491.0 
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Conclusions 



All Anchorage students take both the Anchorage Pre-Algebra Test and the State of 
Alaska Benchmark 2 Math exam in the spring of each year. The Anchorage Pre-Algebra 
exam was developed by Anchorage teachers to help select students for grade 7 Pre- 
Algebra. The State of Alaska Benchmark 2 Mathematics exam was developed to 
establish the extent to which Alaska students have met State benchmarks for mathematics 
performance at the end of grade 6. 

The scores on the examinations are highly related with a correlation of greater than .8 
even though the Anchorage exam consists of all multiple-choice questions while the 
Alaska Benchmark exam includes short answer and extended response questions. 

Examination of the recommendations that might be made for student placement based on 
the two examinations following an Equipercentile equating process showed that the 
recommendations made based on the two examinations would be highly similar. The 
Anchorage Pre-Algebra Test resulted in the recommendation of 77% of the students for 
Pre-Algebra. A slightly higher percentage of students would have been recommended for 
Pre-Algebra (82%) using the State of Alaska Benchmark 2 Mathematics examination. 

Examination of the accuracy of the prediction of placement based on test scores alone 
showed that the Anchorage Pre-Algebra exam that was actually used for placement of 
students was only slightly better in predicting actual placement than the State of Alaska 
Benchmark 2 Examination. The accuracy of predictions for placement for the Anchorage 
exam for Math 7 and for Pre-Algebra was 77% and 67% as opposed to predictions of 
82% and 64% for the State of Alaska Benchmark 2 Math Exam. 

The two examinations appear to be equivalent in their recommendations for placement 
and the predicted accuracy of placement of 2001-2002 students. Examination of the 
letter grades actually earned by students show that there is a moderate association of 6 <h 
grade scores with letter grades for both examinations. However, the average scores on 
either exam consistently and significantly relate to the grades earned by students. 

The evidence of similar recommendations, similar predicted placements, and similar 
patterns of grades earned relative to test scores all support the consequential validity of 
either examination when used as part of the process to select students for Pre-Algebra. 
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Recommendation 



Since the two examinations appear to be highly related and substantially equal as tools 
for predicting the success of students in Pre-Algebra, it is suggested that the Pre-Algebra 
recommendation be based on the Alaska Benchmark 2 Mathematics examination. Use of 
the State examination for placement will do no harm to the recommendation process and 
will reduce the amount of testing required of grade 6 students. 

The use of the State of Alaska Benchmark 2 Math test for Pre-Algebra placement has two 
other potential advantages. First, the use of the State test is recognition of the importance 
of Alaska State Standards and instruction that leads students to high levels of 
performance on the benchmark test as an indicator of master)’ of the state standards. 
Second, the use of the test as part of the Pre-Algebra placement process has a potential to 
serve as a motivator for students who are interested in placement in Pre-Algebra who 
might not otherwise see a practical value in the State test. 
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